Combining Model-Based Meta-Reasoning and Reinforcement Learning for Adapting Game-Playing Agents

Human experience with interactive games will be enhanced if the game-playing software agents learn from their failures and do not make the same mistakes over and over again. Reinforcement learning, e.g., Q-Learning, provides one method for learning from failures. Model-based meta-reasoning that uses an agent’s self-model for blame assignment provides another. In this paper, we combine the two methods. We describe an experimental investigation of a specific task (defending a city) in a computer war strategy game called FreeCiv. Our results indicate that in the task examined, modelbased meta-reasoning coupled with reinforcement learning enables the agent to learn the task with effectiveness matching that of hand coded agents and with speed exceeding that of non-augmented reinforcement learning.